4 research outputs found
Neural Fields for Interactive Visualization of Statistical Dependencies in 3D Simulation Ensembles
We present the first neural network that has learned to compactly represent
and can efficiently reconstruct the statistical dependencies between the values
of physical variables at different spatial locations in large 3D simulation
ensembles. Going beyond linear dependencies, we consider mutual information as
a measure of non-linear dependence. We demonstrate learning and reconstruction
with a large weather forecast ensemble comprising 1000 members, each storing
multiple physical variables at a 250 x 352 x 20 simulation grid. By
circumventing compute-intensive statistical estimators at runtime, we
demonstrate significantly reduced memory and computation requirements for
reconstructing the major dependence structures. This enables embedding the
estimator into a GPU-accelerated direct volume renderer and interactively
visualizing all mutual dependencies for a selected domain point
Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks
Statistical postprocessing is used to translate ensembles of raw numerical
weather forecasts into reliable probabilistic forecast distributions. In this
study, we examine the use of permutation-invariant neural networks for this
task. In contrast to previous approaches, which often operate on ensemble
summary statistics and dismiss details of the ensemble distribution, we propose
networks which treat forecast ensembles as a set of unordered member forecasts
and learn link functions that are by design invariant to permutations of the
member ordering. We evaluate the quality of the obtained forecast distributions
in terms of calibration and sharpness, and compare the models against classical
and neural network-based benchmark methods. In case studies addressing the
postprocessing of surface temperature and wind gust forecasts, we demonstrate
state-of-the-art prediction quality. To deepen the understanding of the learned
inference process, we further propose a permutation-based importance analysis
for ensemble-valued predictors, which highlights specific aspects of the
ensemble forecast that are considered important by the trained postprocessing
models. Our results suggest that most of the relevant information is contained
in few ensemble-internal degrees of freedom, which may impact the design of
future ensemble forecasting and postprocessing systems.Comment: Submitted to Artificial Intelligence for the Earth System
An Emergent Space for Distributed Data with Hidden Internal Order through Manifold Learning
Manifold-learning techniques are routinely used in mining complex
spatiotemporal data to extract useful, parsimonious data
representations/parametrizations; these are, in turn, useful in nonlinear model
identification tasks. We focus here on the case of time series data that can
ultimately be modelled as a spatially distributed system (e.g. a partial
differential equation, PDE), but where we do not know the space in which this
PDE should be formulated. Hence, even the spatial coordinates for the
distributed system themselves need to be identified - to emerge from - the data
mining process. We will first validate this emergent space reconstruction for
time series sampled without space labels in known PDEs; this brings up the
issue of observability of physical space from temporal observation data, and
the transition from spatially resolved to lumped (order-parameter-based)
representations by tuning the scale of the data mining kernels. We will then
present actual emergent space discovery illustrations. Our illustrative
examples include chimera states (states of coexisting coherent and incoherent
dynamics), and chaotic as well as quasiperiodic spatiotemporal dynamics,
arising in partial differential equations and/or in heterogeneous networks. We
also discuss how data-driven spatial coordinates can be extracted in ways
invariant to the nature of the measuring instrument. Such gauge-invariant data
mining can go beyond the fusion of heterogeneous observations of the same
system, to the possible matching of apparently different systems